DOCUMENT RESUME 



TM 026 570 

Kao, Michelle T./ Lehman, James D. 

Scaffolding in a Computer-Based Constructivist Environment 
for Teaching Statistics to College Learners. 

Mar 97 

23p./ Paper presented at the Annual Meeting of the American 
Educational Research Association (Chicago, IL, March 24-28, 
1997) . 

Reports - Research (143) -- Speeches/Meeting Papers (150) 
MFOl/PCOl Plus Postage . 

♦College Students; *Computer Assisted Instruction; 
♦Constructivism (Learning) ; Foreign Countries; Higher 
Education; Hypothesis Testing; Knowledge Level; ♦Learning 
Strategies ; Models ; ♦Statistics ; Teaching Methods ; Transfer 
of Training 
♦Scaffolding 

Scaffolding refers to the instructional support that 
instructors or more skillful peers offer learners to bridge the gap between 
their current skill levels and the desired level. An aspect of scaffolding 
that is often ignored is the fading of support as the learner masters the 
skill. It has been suggested that there is a risk of over-relying on the 
support of integrated media in computer-assisted instruction. A 
three -dimens ion (3-D) model of scaffolding that incorporates level of 
subtask, level of support, and number of repetitions of practice has been 
proposed to vary the technology support systematically in response to the 
learner’s performance. The 3-D contingent scaffolding model was implemented 
in a computer-based instructional program for statistics called "Hypothesis 
Testing- -the Z-Test" in order to establish baseline data for integrated 
media-based instruction or a hypermedia learning environment. The scaffolded 
instruction as evaluated in terms of knowledge maintenance and transfer by 
comparing it to full -support instruction and least -support instruction. 
Findings from 75 college students provide evidence that the scaffolded 
computer-based instruction promoted knowledge maintenance and improved 
independent knowledge application, while promoting learning consistently 
across individuals. Results also show that a dynamic measure of the learner's 
ability is a better predictor of the learning outcome for subjects using this 
scaffolded instruction than static measures. The model provides a systematic 
way to link the concept of scaffolding to integrated media design features 
using both support building and support fading techniques. (Contains 2 
tables, 6 figures, and 17 references.) (SLD) 



ED 408 317 

AUTHOR 

TITLE 

PUB DATE 
NOTE 



PUB TYPE 
EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 

ABSTRACT 



******************************************************************************** 

♦ Reproductions supplied by EDRS are the best that can be made ♦ 

♦ from the original document. ♦ 

******************************************************************************** 




Kao and Lehman 



AERA, 1997 



«v 



1-H 

m 

00 

o 




Scaffolding in a Computer-Based Constructivist Environment 
for Teaching Statistics to College Learners 



U.S. DEPARTMENT OF EDUCATION 
Office of Educational Research and Improvement 
EDUCATIONAL RESOURCES INFORMATION 
/ CENTER (ERIC) 

B^This document has been reproduced as 
received trom the person or organization 
originating it. 

□ Minor changes have been made to 
improve reproduction quality. 



• Points of view or opinions stated in this 
document do not necessarily represent 
official OERI position or policy. 



Michelle T. Kao 
National Dong-Hwa University 
Hwalien, Taiwan 

James D. Lehman 
Purdue University 
West Lafayette, Indiana 



PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL 
HAS BEEN G RANTE D BY 

H /(LH£rLL€ ~T'- 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 










ABSTRACT 

To prevent the user from over-relying on technology support, the researchers 
proposed a 3-D contingent scaffolding model to systematicdly vary the technology support 
in response to the learner’s performance in a learning task consisting of a sequence of sub- 
tasks. The research reports on the implementation of this model in a computer-based 
instructional program on a topic in statistics and a study of its effectiveness with college 
students. 

In this study, the 3-D contingent scaffolding model was implemented in a 
computer-based instructional program, “Hypothesis Testing— the Z-Test,” in order to 
establish baseline data for integrated-media-based instmction or a hypermedia-based 
learning environment. This scaffolded instruction was evaluated in terms of knowledge 
maintenance and knowledge transfer by comparing it to full-support instmction and least- 
support instmction. A secondary interest was to determine whether the predictive utility of 
static (what the learner already Imew) and dynamic (the learner’s ability to learn) measures 
of learners’ ability varied in different instmctional conditions. The dynamic measure was 
indicated by the percentage of successful trials in the learning process. The static measures 
involved seven baseline variables: courses taken, self-rating of skills, and self-rating of 
preference in mathematics/statistics and pretest. 

The findings showed evidence that the scaffolded computer-based instmction 
promoted knowledge maintenance and improved independent knowledge application. The 
scaffolded instmction promoted learning consistently across individuals. However, further 
investigation and evidence is needed to confirm its superiority on knowledge transfer. The 
results also revealed that the dynamic measure of the learner’s ability was a better predictor 
of the learning outcome for subjects using this scaffolded instmction than the static 
measures. In media-assisted instmction, the 3-D contingent scaffolding model provided a 
systematic way to link the concept of scaffolding to the integrated media design features 
using both support building and fading techniques. In statistical education, this model was 
implemented successfully on the topic of hypothesis testing. The results of this study also 
showed that this topic could be taught to college students with low mathematical 
background and statistics beginners. 
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INTRODUCTION 

The term “scaffolding” has been used to describe various instructional techniques 
for use in learning activities that reflect authentic task situations. This technique has also 
drawn great attention from the media researchers for media can provide a more realistic 
learning environment with rich and varied support. One major concept of this technique is 
to enable the learner to engage in out-of-reach activities; having a “knowledgeable other” or 
“more capable peer” to bring the learner along; having something or someone “share the 
cognitive load” (Jackson et al., 1995). However, if we go back to Bruner’s original 
definition of scaffolding (Bruner, 1983), we find there is another valuable key concept of 
this technique that has usually been neglected. That is, this knowledgeable or more capable 
other is not only responsible for building the support in learning activities, but also is 
responsible for fading the support as the learner gradually masters the skill. The Cognition 
and Technology Group at Vanderbilt University (1993) has proposed a risk of over relying 
on the support of integrated media. To educate individuals to be independent and active 
learners, there is a need to emphasize the support-fading component of scaffolding while 
we try to integrate this instructional technique into media assisted instruction. 



BACKGROUND 
Method of Scaffolding 

In the past 20 years, the constmctivist paradigm has gradually come to educational 
research. From the constmctivist perspective, the purpose of education is to cultivate 
independent and self-directed learners. Bmner’s metaphor of scaffolding provides an 
explicit strategy to direct our teaching toward this end. 

Scaffolding refers to the interactional support that instmctors or more skillful peers 
offer learners to bridge the gap between their current skill levels and a desired skill level. 

In the process, the amount of support is gradually withdrawn as the learners become more 
proficient. Ultimately, the learners can complete tasks on their own (Greenfield, 1984). 
The literature review suggests that scaffolding can enhance comprehension, improve 
independent learning and application, and promote knowledge transfer (Bmner, 1983, 
Greenfield, 1984, Cazden, 1988). Evidence of these advantages has been found in many 
studies in the field of language and cognitive development (Boyle & Peregoy, 1990; Day & 
Cordon, 1993; Gallimore, T^arp, & Rueda, 1989; Smolucha & Smolucha, 1989). 

At the same time, limitations of scaffolding have also been pointed out. TWs 
technique has been criticized for its lack of discussion concerning development of the 
expert’s role in providing the novice with assistance (Gaffney & Anderson, 1991). Its 
implementation has also been criticized for not being able to capture the challenge of 
responding to the diversity of children’s intentions in classroom teaching (Dyson, 1990). 

Scaffolding in Computer-Based or Integrated-Media-Based Instmction 

Computers have introduced unprecedented levels of autonomy into education. The 
processing and integrating capabilities of computers can create a realistic, interactive, 
support-rich, and individualized learning environment. These characteristics may 
overcome the limitations of scaffolding and ease the implementation of this instructional 
technique. Several researchers have developed scaffolded computer-based instmction, 
integrating more than one medium to support learners’ knowledge constmction in authentic 
learning activities (Jackson et al., 1995; Steiner & Moher, 1994). However, the discussion 
has focused mainly on the software development and the support-building models. Only a 
srnall amount of qualitative evaluation data have been reported. The support-fading element 
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of scaffolding was not applied in these studies. Therefore, scaffolded instruction has been 
not only difficult to build, but also extremely difficult to evaluate. 

Lieberman and Linn (1991) contended that scaffolding is one of the several ways 
computers can be used to encourage students to be self-directed. This can be attributed to 
the support-fading characteristic of scaffolding. However, to use this skill, the major 
challenge is to determine when the support should be faded and how much the support 
should be reduced at the time. This requires us to ascertain the learner’s mastery level at 
any point of the scaffolded learning process. 



LINKING SCAFFOLDING TO INTEGRATED-MEDIA DESIGN FEATURES 

In this research, we sought an operational definition of scaffolding. Based on the 
definition, we proposed a 3-D scaffolding model which provides a systematic way to apply 
scaffolding in computer-based or integrated-media-based instruction. 

The Elements of Scaffolding Relevant to Integrated-media Design 

To link the concept of scaffolding to computer-based instruction (CBI) design 
features, it is important to identify some basic elements of scaffolded instruction which are 
especially relevant to CBI design. To illustrate the features of those elements, the metaphor 
of an adult aiding a toddler (Cazden, 1988) will be used as an example. 

1. Hierarchical component skills 

In the metaphor of scaffolding, the acquisition of skill is conceived as a hierarchical 
program in which component skills are combined into “higher skills” by appropriate 
orchestration to meet new, more complex task requirements (Bruner, 1973). Therefore, 
the crucial task in scaffolding often possesses a variety of relevant components (usually in 
an appropriate serial order) necessary to achieve a particular end. It is die instructor’s 
responsibility to decompose the final task into hierarchical component skills based on the 
nature of the task and the learner’s ability. In the example of an adult aiding a toddler, the 
child must learn how to balance on his feet before taking his first step, must learn how to 
balance on his first step before stretching out for the second. 

2. Decreasing support levels 

Scaffolding emphasizes the support that the instructor can provide the learner to 
reach beyond his/her current skill level. The highest level of support is the situation where 
the instructor completes the task as a demonstration and the learner takes no responsibility 
for the current task. The lowest level of support is the situation where the instructor takes 
no responsibility for the task and the learner completes the task on his/her own. Between 
the hipest and lowest level of support, the instructor shifts a part of the responsibility to 
the learner by reducing the amount of support one level at a time. In the sense of support 
withdrawn, Ae instructor must recognize what kind of support is crucial in the learning and 
classify the support into decreasing levels. In the example of an adult aiding a toddler, the 
levels of support could be: helping by holding two hands, helping by holding one hand, 
helping by holding one finger, etc. 

3. Repetitive authentic practice 

The scaffolding should be performed in the context of the practical work 
environment. In the environment, the learner can practice by taking part in the expert 
behavior in a realistic setting. Each practice involves a full performance of the task. 
Besides, the underlying structure of the practice must be repetitive so the experience at one 
point in task mastery could potentially h« applied to later activity. Therefore, the instructor 
has to set up a sequence of authentic practice involving the performance of the same skills. 
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In the walking example, the child actually experiences the practical task, walking. The 
practice of walking involves the same component skills and the practice can be repeated in 
different realistic settings: walking on carpet, walking on solid ground, walking on sand, 
and so forth. 

4. Ongoing assessment 

In the process of scaffolding, the support level should be adjusted based on the 
learner’s current skill level. Therefore, the instructor must measure the learner’s progress 
against the global picture of the task and make corrections when needed. Usually, the 
completion of each component skill is the right time for a progress assessment. In the 
wallang example, the adult observes and measures the child’s progress with each baby 
step. If holding one hand of the child doesn’t keep his balance, the adult gives him the 
other hand right away. If the child walks well by holding one hand, the adult will consider 
letting him walk by holding only one finger. 

Therefore, an operational definition of scaffolding could be as follows. The 
instructor or the more skillful peer decomposes the task into hierarchical sub-tasks, 
classifies the amount of support in decreasing levels, and sets up repetitive authentic 
practice. The practice begins with the highest level of support and the lowest level of sub- 
task. With the completion of each sub-task in the practice, the instructor measures the 
learner’s performance and judges the level of support he/she should provide and lets the 
learner perfect the component sub-task that he/she can manage. 

The Three-Dimension Contingent Scaffolding Model 

Wood et al. (1976) and Day and Cordon (1993)’s studies have suggested the 
“contingent scaffolded instruction,” emphasizing the addition of a pattern of responses for 
the withdrawal of the support. The pattern involves characterizing the instructor’s 
intervention by the level of abstracmess and varying the level systematically in response to 
the learner’s performance. Based on the first two elements of scaffolding discussed above, 
there are two types of intervention that the instructor can provide in scaffolded instruction. 
The first type involves the control of the task complexity by simplifying the task itself. The 
second type involves the control of the task complexity by providing additional support. 
Most studies (Day and Cordon, 1993; Wood, Bruner, &. Ross, 1976; Wood, Wood, & 
Middleton, 1978) have focused on the second t^ of intervention. In this study, the 
researchers proposed a 3-D contingent scaffolding model, considering both types of 
interventions to apply the contingent scaffolded instruction in a learning task consisting a 
sequence of steps. 

Figure 1 is a graphic illustration of this model for a target task consisting of 4 hierarchical 
sub-tasks and 4 levels of support. The first two elements of scaffolding actually describe 
16 intervention patterns that the learner might encounter at any time depending on the 
learner’s current skill level. The process of scaffolding starts with the completion of the 
full task (from sub-task 1 to sub-task 4) at the highest level of support (support level 1) and 
ends with the completion of the full task at the lowest level of support (support level 4) 
through a sequence of repetitive authentic practice, the third element of scaffolding. The 
whole process of scaffolding is performed according to the following rules. 

1 . Each practice involves a full performance of sub-tasks from the first to the last. 

2. The first practice starts with the highest level of support. After that, each practice starts 
with the level of support which is one level lower dian the latest one used in the 
previous practice. 
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3 . At the current level of support, the learner has the chance to work on the following sub- 
tasks unless he/she encounters any difficulty. 

4. In each sub-task of the practice, if the learner encounters any difficulty, the support 
level is increased by one until reaching the highest level. 

The fourth element of scaffolding, ongoing assessment, is actually embedded in the 
rule 4. To see if the learner encounters any difficulty, an ongoing assessment must be 
performed at the end of each sub-task. 



level of sub-task 




Figure 1. Graphic illustration of the 3-D contingent scaffolding model. 



This model of scaffolding describes the learning process as a journey in the space 
of the three dimensions: level of sub-task, level of support, and numter of the repetition of 
the practice. The first practice is presented as an example in which the instructor takes full 
responsibility for the task (full support is provided). The last practice is a completely 
successful examination where the learner takes the full responsibility of the task (the least 
support is provided). Any practice in between either is an unsuccessful performance or a 
successful performance but with some level of support. Along the path of the journey, 
each cubic block is a unit of assessment of the learner’s progress. Since this model applies 
scaffolding in a systematic way, it is especially useful for the computer or integrated media 
based instruction. 
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AN EXAMPLE OF IMPLEMENTATION: “HYPOTHESIS TESTING- THE Z-TEST’ 

Based on the 3-D model, the software, “Hypothesis Testing— the Z-test,” was 
developed. In this section, we describe the program, “Hypothesis Testing— the Z-test,” 
and explain how the 3-D scaffolding model was implemented in this program. 

The Motivation of the Task 

Statistics is interesting and useful because it is a means of using data to gain insight 
into real-world problems. However, almost every statistics beginner experiences 
difficulties understanding the topic. Almost every statistics instructor admits that it is not 
an easy subject to teach. 

As the continuing revolution in computing relieves the burden of calculating and 
graphing, there is an emerging consensus among statisticians that statistical education 
should focus on data and on statistical reasoning rather than on either the presentation of as 
many methods as possible or the mathematical theory of inference (Moore and McCabe, 
1993). This consensus has the following influence on statistical education. 

1. Applied Statistics should be taught in the context of real-world problems. 

Statistics takes real life patterns in real life phenomena and tests them quantitatively. 
Bridging the gap between reality and numbers is the hardest part. Not all numbers are data. 
The number 10.3 acquires meaning only when we are told that it is the birth weight of a 
child in pounds or the percent of teenagers who are unemployed. That is the context that 
makes the numbers meaningful. Therefore, the examples and exercises should be 
presented in the context of real-world problems. 

2. Applied Statistics can be taught to an extremely diversified audience. 

Because the statistics is useful for scientific investigators in many fields, the 
students sitting in a introduction course of applied statistics might have varied background. 
However, if the student has no fear of statistics, wants to learn, and possesses. a certain 
amount of mathematical maturity, the varied backgrounds need not detract from the success 
of the course. In fact, many times this diversity may make students recognize similarities 
among problems from different fields of application. This gives them a broader scientific 
approach to statistical problems than providing them only with familiar examples. 

3. Applied Statistics can be taught at a low mathematical level. 

Since it is the statistical reasoning that should be focused upon instead of the 
statistical computation, even students with low mathematical level can learn statistics. 
Therefore, the students of applied statistics can range in mathematical competence from 
Ph.D. candidates in mathematics to students who have had only high school algebra. 
Actually, the learners need only a working knowledge of algebra. Being able to read and 
use formulas would be enough. 

4. Applied Statistics can be taught through the applications of a set or a sequence of rules. 

Applied Statistics is actually the application of a sequence of rules or principles 
which is set up based on certain mathematical proofs or the experience of prior statistical 
experts. The learners do not have to understand the mathematical proof behind each rule. 
However, it is important for them to understand the expert reasoning behind those rules. It 
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is not only logical but also conceptual. To understand the statistical reasoning, the learner 
has to work and communicate with the expert to cany out the rules in realistic problems. 

Based on the above discussions, we decided to develop a scaffolded computer 
based instruction in the context of teaching basic applied statistics. The very basic test of 
statistical hypothesis testing, the Z-test, was selected. The task was selected with several 
objectives in mind. First, it had to be challenging to the learner while also proving 
sufficiently complex to ensure that his/her comprehension and problem-solving skill over 
time could develop. Second, it had to be “feature rich” in the sense of possessing a variety 
of relevant components. Third, its underlying structure had to be repetitive so the 
experience at one point in task mastery could potentially be applied to later activity. 

This subject matter involves a sequence of problem solving steps and decision 
making. It naturally provides the chance for assessing users' ongoing learning as well as 
scaffolding through program support. Therefore, the program instruction is presented as a 
series of problems involving applying the Z-test. 

Implementing the 3-D Contingent Scaffolding Model in “Hypothesis Testing- the Z-Test” 

Table 1 summarizes the four elements of the scaffolding model and briefly 
describes how each was implemented. 



Table 1 






Elements of Scaffolding 


CAI Design Features 


1. Hierarchical component skills 


Four serial steps to carry out the Z-test 


2. Decreasing support levels 


Four levels in amount of support 


3. Repetitive authentic practice 


Up to 20 authentic problem scenarios 


4. Ongoing assessment 


Performance is judged at the end of each step 



1. Hierarchical component skills 

According to the common practice of hypothesis testing, the program decomposed the 
task into four sub-tasks: state the hypotheses, find the critical value, compute the test 
statistic, and make the decision. 

2 . Decreasing support levels 

In considering the support which can be provided via integrated-media and the nature of 
the task, three types of support were selected to be used in the scaffolded instruction. 
From concrete to abstract, the three types of support were: visual support, verbal 
support, and symbolic support. 

The actual environment of the lesson is displayed in Figure 2. It shows how 
the visual, verbal, and symbolic support appeared on the program screen. 

1. Problem Area 

The Problem Area showed the current problem scenario that the student was 
working on. 
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2 . Instruction Area 

The Instruction Area provided the instruction or hints to solve the current step of the 
problem. It functioned as the verbal support. 

3 . Figure Area 

The Figure Area illustrated the current step of the problem in a graphic form based 
on the learner’s selection of the answers. That was where the visual support took 
place. 

4 . Answer Area 

The Answer Area provided the answer or requested the learner to provide an 
answer for the current step after prompting with symbolic support. 




Figure 2. The environment of the lesson. 
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Based on the three types of support described above, the levels of support can 
be classified and ranked by the amount of support (how many types of support) 
provided at the time. These are described below. From level 1 to level 4, die next most 
concrete type of support was withdrawn from the previous level of support. 

Level 1: Full Support 

The instruction demonstrated the steps to solve the problem in detail with 
visual, verbal, and symbolic information. Figure 3 is an example of the full 
support screen. 

Level 2: Visual, Verbal, and Symbolic Support 

Instead of providing the answer with detailed explanation, the instruction only 
provided the visual and verbal hints to the current problem step and requested 
the learner to give the answer after the symbolic prompts. An example of the 
screen with level 2 support is provided in Figure 4. 





Hypothesis Testing 




1. Rats th^ ate raised in a laboratory environment have a mean life span of around 24 months. A 
sample of 36 rats reared to adulthood in a germ-free environment had life spans with a mean of 27. 
Assuming a population standard deviation of 6 months., does this type of rearing increase the fife 
span of the laboratory rats to a point that is enough to refect the null hypothesis at 0.05 level of 
significance? (McGhee^ 1985 . p331« Exaraple 9.13) 


ILext 


From the problem statements, we expect the mean p of the 
population from which the sample was drawn will be greater than 24. 
Therefore, the null hypothesis Ho and the alternative hypothesis Hi, 



End 



STEP 1 LEVEL 1 




written in symbols, are 
HO: u = 24 
H1:u>24. 

A one-tailed test to the 
right is required. 



Figure 3. An example screen with level 1 support. 
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Hypothesis Testing 



New 


2. In several stuifies it has been reported that the natural age at menopause of non smoking women 
is around 50 years. In a study entitled Cigarette Smoking and Natural Healthy a sample of 49 heavy 
smokers had a mean age at menopause of 48 years. Assuming a population standard deviation of 
5.6 years« test the hypothesis that cigarette smoking is associated with early onset of menopause 
with 0.05 level of significance. (McGhee« 1985« p332« Example 9.14) 




The first step of the hypothesis testing is to state the Null and 
Alternative H^otheses. What hypotheses can you state for this 

problem? of the ttuttons to decide the direction of the attemative hypothesis. 




Figure 4. An example screen with level 2 support. 



The Figure Area in Figure 4 shows an initial form of the graphic illustration of the current 
step. Step 1. Based on the learner’s selection of the answers, the corresponding 
information was added to the graph. 

Level 3: Verbal and Symbolic Support 

The information in the figure area disappeared. Only the verbal hint was 
provided, telling the learner some specific information to look for and asking 
him/her to provide answers after the symbolic prompts. 

Level 4: Symbolic Support 

The instruction took away the next most concrete support from the screen. 
Therefore, only the symbolic prompts were provided and the learner was 
required to give the answer. 

3 . Repetitive authentic practice 

The problems were adapted from many statistics text books, covering the real problems 
occurring in many disciplines: agriculture, biology, engineering, pharmacy, business, 
and the social sciences. 

4. Ongoing assessment 

At the end of each step, the learner’s answer was judged for correctness. This 
judgment decided the next intervention pattern the learner was to encounter. 
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This specific scaffolded computer-based instruction, “Hypothesis Testing— the Z 
test,” was expected to be an effective and powerful instructional program, inheriting the 
advantages and minimizing the limitations of scaffolding. In this study, this scaffolded 
instruction was evaluated in terms of knowledge maintenance and knowledge transfer by 
comparing it to the full-support instruction (in which the full support was available all the 
time), and the least-support instruction (in which only the symbolic support was provided). 
A secondary interest was to determine whether the predictive utility of static (what the 
learner already knew) and dynamic (the learner's ability to leam) measures of learners' 
ability varied in different instructional conditions. 



EVALUATION OF "HYPOTHESIS TESTING- THE Z-TEST " 

"Hypothesis Testing— the Z-Test" was evaluated to address the following research 

questions: 

The effect of the instructional method: 

• Would the availability of full support in the computer-assisted learning hamper learners' 
independence in knowledge application? 

• Would scaffolded instruction enhance knowledge maintenance in a computer-assisted 
learning environment? 

• Would scaffolded instruction promote knowledge transfer in a computer-assisted 
learning environment? 

The effect of the static measure of the learner’s ability ( what the learner had already 

learned): 

• Would learners’ background, confidence, and preference in statistics and mathematics 
influence their knowl^ge maintenance and transfer? 

The effect of the dynamic measure of the learner’s ability ( the learner’s ability to leam): 

• Would the learners who showed higher ability to leam in the problem-solving process 
maintain and transfer the strategy better? 

The relationship between the static/dyruimic measure and the instructional methods: 

• Would the predictive utility of static measure (what the learner had already learned) and 
dynamic measure (the learner’s ability to leam) vary in the three support conditions: 
scaffolded, full support, and least support? 

The relationship between the static measure and the dyrumiic measure: 

• Would learners' prior knowledge influence their learning ability in the task? 
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The Experiment 
Research Design 

Based on the research questions described above, a baseline-posttest control group 
design was implemented using the computer program, “Hypothesis Testing— the Z-test.” 
The research questions regarding the main effects of the instructional method, the static 
measures of the learner’s ability, and the dynamic measure of the learner’s ability were 
examined under a regression model (more specifically, an analysis-of-covariance model). 
The research questions regarding the relationships between each other of the three major 
variables were examined using partial correlation and correlation analyses. 

Regressor Variables 

There were three types of the regressor variables involved, representing the three 
different interests of the study. 

1 . Treatment Variable- Support Condition 

The effect of this type of variable was the major interest of this study. It was 
manipulated by the computer-based instruction. The design framework of the 
scaffolded computer-based instruction program, “Hypothesis Testing— the Z-test,” was 
discussed in detail in the previous section. Part of the design framework is reviewed 
here: 

From concrete to abstract, the three types of support used in the program were visual 
support, verbal support, and symbolic support. The levels of support were classified 
and ranked in the amount of support (how many types of support were provided at the 
time). From level 1 to level 4, the next least abstract type of support was withdrawn 
from the previous level of support. They were: 

Level 1: Full Support; 

Level 2: Visual, Verbal, and Symbolic Support; 

Level 3: Verbal and Symbolic Support; and 

Level 4: Symbolic Support 

Based on the design framework, the program was created in a powerful and 
flexible programming language. Visual Basic 3.0 for Windows. In order to evaluate 
the effectiveness of this scaffolded instruction, two versions with different non- 
scaffolded support conditions were adapted from the original program, one with the 
full-support condition and the other with the least support condition. The three 
different instructional conditions are described below. 

Scaffolded Condition (adjusted level of support among 1, 2, 3, and 4 in an orderly 
fashion depending on the learner’s current ability) 

Based on the four levels of support, the program started from the level of full support. 
The amount of support was reduced by one level in the next problem as the user 
demonstrated the ability to solve the present problem at the current level of support. 
Once the user encountered difficulty at any step of the problem-solving process, the 
amount of the support would be increased by one level for the current and the rest of 
the problem steps immediately. The teaching process proceeded until the user could 
solve two consecutive problems with the lowest level of support. 

Full Support Condition (swung between levels of support 4 and 1) 

This condition represents the support condition which has been used in many 
computer-assisted learning environments. The program first demonstrated the complete 
problem solving procedure in a full-support example. Then it gave another problem 
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with the least amount of support (i.e. symbolic support) as a test. Full support was a 
screen option which could provide immediate help if the user encountered any difficulty 
in the problem-solving process (see Figure 5). If the user did not ask for full support 
but made any mistake in any step of the problem solving procedure, the fuU support 
would be provided starting from the current problem step. The current problem would 
then be solved as an example, and the next problem would be presented with the least 
level of support as a test again. That is, the program support swung between the levels 
of full support and least support. The teaching process proceeded until the user could 
solve two consecutive problems with the least amount of support. 



Hypothesis Testing 






2, In sever at studies it has been reported that the natural age at menopause of non smoking women 
is around 50 years, in a study entitled Cigarette Smoking and Natural Health, a sample of 49 heavy 
smokers had a mean age at menopause of 48 years. Assuming a population standard deviation of 
5.6 years, test the hypothesis that cigarette smoking is associated with early onset of menopause 
with 0.05 level of significance. (McGhee, 1985, p332. Example 9.14) 



Next 



Click one of the buttons to decide the direction of the alternative hypothesis. 



flelp 



lEndl 



STEP 1 LEVEL 4‘ 



HO: u = 50 
H1:u[]50 



Figure 5. Full support was an on-screen option, “Help”, in the full support condition. 



Least Support Condition (used level of support 1 once and thereafter level of support 4) 
This condition of program support represented the traditional teaching method which 
has been used in many statistic textbwks. It served as a control condition. As in the 
other two treatment groups, the program first explained the four steps of hypothesis 
testing procedure in a sample Z-test problem thoroughly (i.e. with full support). 
However, in the following problems, only the least amount of support (i.e. symbolic 
support) was provided. The user’s responses in each step were judged for correctness 
at the end of each problem. If any mistake occurred in any step of the problem solving 
procedure, the correct answers were provided without explanation (see Figure 6). The 
process proceeded until the user solved two consecutive problems. 

All conditions of the instructional program accessed the problems from a data file which 
contained 20 problems involving Z-test about one-tailed single sample mean. 
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= 


Hypothesis Testing 



New 



N«xf 



5. The dischaige teoipefatiJie of coolant watei from a nuclear reactor is stipulated to have a mean 
of 45. An environmental group believes it to be higher and on 36 different occasions obtains 
temperature rearfings that have a mean of 46.8. Assuming a population standard deviation of 3.6« 
carry out the test Use 0.025 as the level of significance. (McGhee^ 1985« p333« Exercise Set 9.3« 
PartA, 87) 



End 



Z=3 
Za= 1.64 

Click one of the b 
STEP 4 LEVEL ^ 



Please check your answers: 



The answers should be: 

HO: u = 45 
Hl:u>45 
Zo.= 1.96 
Z - 2.99 

Decision: reject HO 





or: 







ect 



HO 



Accept 



Figure 6. Correct answers were provided without explanation in least support condition. 



2. Static Measure of the Learner’s Ability (what the learner already knew) 

Related baseline factors in mathematics and statistics were considered. They 
were: courses taken in Mathematics (MATH_C), courses taken in Statistics 
(STAT_C), self-rating of mathematical skills (MATH_S), self-rating of statistical skills 
(STAT_S), mathematics preference (MATH_P), and statistics preference (STAT_P). 

In addition, the pretest score was also collected as a baseline measure of students' prior 
knowledge about the topic to be taught. 

3 . Dynamic Measure of the Learner’s Ability (the learner’s ability to learn) 

This measure was represented by the percentage of the successfol trials where 
each problem step was considered as a trial after the first problem. For the scaffolded 
condition, as long as the learner provided a correct answer to the problem step, it was 
counted as a successful trial, no matter which level of support was provided. 

However, this indicator was not free from the influence of the treatments, the 
instructional condition, because: 

• Students in the least support condition had the chance to try and guess on the 
subsequent problem steps although a wrong answer had been provided. 

• Students in the scaffolded condition had the chance to retry on the same problem 
step and try on the following problem steps at higher support levels. 




14 



Kao and Lehman 



AERA, 1997 



When a concomitant variable is affected by the treatment, covariance analysis 
will remove some (or much) of the treatment effect on the dependent variable (Neter, et 
al., 1985). To address this problem, two runs of the covariance analysis were 
conducted, one with dynamic measure, the other without. Therefore, the results of the 
two analysis could be compared and a more critical analysis could be made. 



Response Variables 

Two response variables were involved in this study. They were students’ 
percentage correct on the maintenance posttest and transfer posttest. 



Subjects 

Subjects were 75 students enrolled in an introductory course of educational 
psychology at a major midwestem university in the fall semester of 1995. The majority of 
the students were juniors and sophomores in educational majors. 

Prior to the study, the concepts of the normal distribution, standard deviation, and 
the Z-score were covered in the first 6 weeks of the course. Therefore, the materials of 
instructional program were relevant and appropriate to the students. The participation of 
the study was counted as two-hour research credits. Students were invited on a volunteer 
basis. Subjects were then randomly assigned into the three condition groups. 

Among the 75 students, 3 students scored more than 80 percent of the total score on 
pretest. They were dropped from the analysis. Therefore, only 72 students (24 in full 
support group; 26 in scaffolded group; and 22 in least support group) were included in the 
analysis. 



Instruments 

All the tests were administered in paper-and-pencil format with a calculator and a 
copy of the Z-score table provided. 

Pretest and Maintenance Posttest 

Each test consisted of three problems involving a one-tailed Z-test of single sample 
means. Each problem consisted of four steps and each step involved at least three different 
subconcepts. A total score of 48 was possible. 

Transfer Posttest 

The transfer posttest contained one one-tailed and two two-tailed Z-test problems of 
single sample means. Each problem consisted of four steps and each step involves at least 
three different subconcepts. A total score of 48 was possible. 

All the test problems were adapted from statistic textbooks and previewed by two 
experts in statistics. 



Procedure 

The experiment was conducted in the seventh to tenth weeks of the fall semester of 
1995. It consisted of two sessions with 7 to 10 days in between. For better control of the 
experiment, the first session was conducted by one of the researchers. 
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Session One 

Each participant first completed a participant survey at the beginning of the first 
session of the experiment. Then the participant was informed of the session agenda and the 
expected time spent on each activity. After that, the researcher checked the participant’s 
ability to use a computer mouse and demonstrated the skill if necessary. Before working 
on the computer, the participant was provided with a hand-held calculator, a Z-score table, 
and a couple of worksheets for notet^ng and calculation. The participant could keep and 
use these tools for the entire session. The participant then started to work on computer- 
based pre-instruction for about 20 minutes. The pre-instruction reviewed all of the 
prerequisite skills and knowledge needed for the major instruction. Then, for 10 minutes, 
the participant completed a self-evaluation over the key concepts learned in the pre- 
instruction by working on the paper-and-pencil prerequisite test. A key with detailed 
explanation was provided. After the prerequisite test, the participant was given a three- 
problem pretest. The participant was told at the beginning that he/she was not expected to 
know the answers to these problems. If having spent more than three minutes on one 
problem but was unable to proceed, he/she could quit the pretest there. The expected time 
for the pretest was 10 minutes. After completing Ae pretest, the participant went back to 
the computer and worked on the major instruction for 45 minutes. 

The major instruction started with a short introduction to the lesson environment. 
After that, the program demonstrated a step by step procedure to carry out a Z-test in a 
problem example. Then, the participant encountered a series of similar problems requiring 
him/ her to provide the answers step by step. Along with these probleihs, one of the 
program support conditions (scaffolded, full support, or least support) was applied, 
depending on which condition group the participant had been assigned into. The program 
could not be terminated until the participant solved two consecutive problems with the least 
amount of support. However, the researcher would ask the participant to stop if he/she had 
spent more than 45 minutes on the major instruction and not finished. During the 
instruction, the participant could use the Z-score table at hand or a version of the table 
available on the computer. The participant’s learning path and responses were recorded in 
a computer file for later retrieval. The number of problems the participant went through 
either to mastery or by reaching the time limit was noted. 

Session Two 

The second session was held about 7 to 10 days after the first. Each participant 
first was asked to complete the three-problem maintenance pbsttest for about 20 minutes. 
After the test, a one-page instruction on two-tailed Z-test was provided. After 10 minutes, 
the instruction sheet was taken away, and the three-problem near transfer posttest was 
administered. Again, the subjects could work on the test for 20 minutes. Upon completion 
of the test, the solutions to all the test problems were provided to the subject. 

Data Analysis and Results 



Covariance analyses were used to examine the main effects of the instructional 
method, the static measures of the learners’ ability, and the dynamic measure of the 
learners’ ability. Partial correlation and correlation analyses were used to examine the 
relationships among the static measures, the dynamic measure, and the treatment. 

Results of Covariance Analysis 

The Effect of the Instmctional Method 

1 . The results of covariance analysis indicated a significant group difference on the 
maintenance posttest (F = 4.03 w/o and F = 3.98 w/ dynamic measure, p < .05). The 
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post hoc pairwise t-test showed no significant difference between the full support group 
and the least support group at the .0167 experimentwise or the .05 comparisonwise 
significance level. Therefore, the availability of full support in the computer-assisted 
instruction did hamper learners’ independent problem-solving on the maintenance 
posttest. 

2. The post hoc pairwise t-test showed that students in the scaffolded group performed 
significantly better than the other two groups on the maintenance posttest at the .0167 
experimentwise significance level (t = 2.75 w/o and t = 2.82 w/ dynamic measure). 
Therefore, the scaffolded computer-assisted instruction did enhance learners’ 
knowledge maintenance. 

3 . No significant group difference was found on the transfer posttest at the p = .05 level. 

The Effect of the Static Measure of the Learner’s Ability 

1 . Only the pretest produced a significant effect on the maintenance posttest (F = 6.53 w/o 
and F = 5.88 w/ dynamic measure, p < .05). 

2 . The transfer posttest score was significantly influenced by the pretest (F = 6.36 w/o 
and F = 6. 14 w/ dynamic measure), the statistics courses taken before (F = 4.76 w/o 
and F = 4.74 w/ dynamic measure), and the learner’s self-rating of mathematical skill 
(F = 6.97 w/o and F = 5.57 w/ dynamic measure) at the .05 significance level, 
suggesting that the learners who got higher scores on pretest, showed more confidence 
in mathematical skills, or took more and higher-level statistical courses performed better 
on the transfer posttest. 

The Effect of the Dynamic Measure of the Learner’s Ability 

1 . The effect of the dynamic measure was marginally significant on the maintenance 
posttest at the .05 level of significance (F = 3.89, p = .0531), indicating that the 
learners who showed higher ability to leam in the problem-solving process did maintain 
the strategy better. 

2 . The effect of the dynamic measure reached the .05 significance level on the transfer 
posttest (F = 3.89). Therefore, the learners who showed higher ability to leam in the 
problem-solving process also transferred the strategy better. 

Results of Partial Correlation and Correlation Analysis 

The Relationship Between Static/Dynamic Measure and Treatments 

The predictive utility of the static and dynamic measures varied in the three support 

conditions: scaffolded, full support, and least support. 

1 . The static measure predicted the students’ posttest scores well in the full support 
condition (p£ = .5 1 for maintenance and pr = .50 for transfer, p < .05). 

2. The static measure predicted the students’ posttest scores best in the least support 
condition, but only the results for the transfer posttest were statistically significant (p£ = 
.58, p < .01). 

3 . The dynamic measure predicted the students’ posttest scores best in the scaffolded 
condition (p£ = .40 for maintenance and p£ = .45 for transfer, p < .05). 
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The Relationship Between Static Measure and Dynamic Measure 

There was a strong relationship between the static measure and the dynamic 
measure, r = .32, p < .01. This was especially true in the least support group, r = .63, g < 
.01. This result suggests that the students’ prior knowledge did influence their ability to 
learn, especially in the least support condition. 

Discussion and Conclu.sions 



The Instructional Methods 

The results of this study confirmed the concern proposed by the Cognition and 
Technology Group at Vanderbilt University (1993) that individuals might over-rely on 
support in a computer environment. The availability of full support in the computer- 
assisted instruction did hamper learners’ performance on the maintenance posttest. 
Although the group difference on the transfer posttest did not reach the level of statistical 
significance, the full support group got the lowest mean score. This suggests that students 
getting the full support all of the time might have over-relied on the available support. 
Therefore, they might have failed to develop their own internal mental model-building 
skills, so that they did not perform well when they encountered a similar but new learning 
situation. Also, the full support group had the largest standard deviation on both posttest 
scores. This means that the effect of this instructional method tended to vary by individual. 
Students’ performance in the fuU support condition may have been more related to their 
original background knowledge than to instructional factors. On the other hand, the results 
of the study showed that scaffolded computer assisted instruction was superior to the non- 
scaffolded instruction in enhancing knowledge maintenance and reducing individual 
differences. 

Although the findings confirmed the superiority of the scaffolded instruction, two 
results differed from the expectations. They were: 

1 . All the three groups had better performance on the transfer posttest when compared to 
the knowledge maintenance posttest. No statistically significant group differences were 
found on the transfer posttest. 

2 . The least support group (the control group) got a higher mean score than the full 
support group on the transfer posttest, although this difference did not reach the 
statistical significance level. 

The differences between the maintenance and transfer posttests provide some 
possible explanations of the first unexpected result. 

1 . First of all, the maintenance posttest tested for knowledge retention. The transfer 
posttest, on the other hand, tested for immediate learning; memorization may have 
played a role in students' superior performance on the transfer posttest. 

2. The transfer posttest used two-tailed tests which involved the same four steps as the ■ 
one-tailed tests. Further, the transfer posttest was administered right after the 
maintenance posttest. The students had just spent mental effort trying to access the 
retained knowledge from their own mental models. The effect of the post-instruction 
therefore might have been in-time and significant. 
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3 . The two-tailed tests did not involve deciding the direction of the alternative hypothesis. 
However, that’s a crucial concept in the one-tailed tests. Many students indicated that 
they felt the two-tailed cases were easier than the one-tailed cases. 

Beyond the above factors, there is still another possible explanation. That is the 
treatment condition may promote direct learning but not transfer. Although the scaffolded 
group got the highest mean score on the transfer posttest, the evidence was not strong 
enough to conclude that this scaffolded computer-based instruction can promote knowledge 
transfer. It still requires further research to verify this point. 

To provide possible explanations for the second unexpected but interesting finding, 
an examination of the differences between the natures of the three instructional methods is 
required. Because the 3-D contingent scaffolding model introduced one more dimension 
into the scaffolding model by dividing the target task into a sequence of sub-tasks, the 
complexity of the group differences was increased. Table 2 compares the three 
instructional methods from three different aspects: chance to retiy the same step, chance to 
guess on the following step, and immediate feedback on correctness. 



Table 2 

Group Differences on Chance to Retry. Chance to Guess, and Immediate Feedback 





Full Support 


Scaffolded 


Least Support 


Chance to Retry the Same Step 


N 


Y 


N 


Chance to Guess on the Following Step 


N 


N 


Y 


Immediate Feedback on Correctness 


Y 


Y 


N 



Note. N = no. Y = yes. 



The influence might be significant whenever the student provided a wrong answer: 

1 . In the full support condition, full support was provided right away whenever the 
learner failed to answer correctly. However, once the student realized that he/she had 
provided a wrong answer, the student tended to skip the explanation and answers to the 
rest of the problem steps and tried the same step in the next problem. Therefore, 
students might only have seen broken pieces of the statistical reasoning and hence lost 
the whole picture of the problem-solving procedure. 

2. On the other hand, students in the least support group did not know they had made a 
mistake until end of the current problem. In this way, the learner more likely generated 
his/her own reasoning of the problem-solving procedure from the first to the last steps 
without any interruption. Besides, since the learner only received the least level of 
support, the learner had to spend more mental effort to figure out the answer to each 
problem step. 

3 . Students in the scaffolded condition had the chance to retry the same problem step with 
a higher level of support right away following an error. The learner not only could 
work on the same step that he/she had focused his/her interest on but also had more 
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support and therefore a better chance to succeed. Students had to spend a fair amount 
of the mental effort and still saw the whole picture of the problem-solving procedure. 
This might also be one of the reasons that made the scaffolded group proved to be 
superior to the others. 

The Static Measure of the Learner’s Ability 

The results of the covariance analysis showed an influence of the pretest 
consistently across posttests, indicating that the pretest score was a very good predictor of 
the posttest scores. However, most of the students did not have any prior knowledge of 
the target learning task (applying the Z-test) except for the very basic statistics concepts 
taught in the intr^uctory course (such as normal distribution and standard deviation). The 
results revealed that the understanding of these very basic concepts (which was reflected on 
the pretest score) might actually improve the learner’s statistical reasoning on the learning 
task (which was reflected on the posttest scores). 

The results showed no significant effect on the maintenance posttest of any of other 
baseline variables. Since the maintenance posttest score was not affected by the learner’s 
mathematical background, students with low mathematical skills could benefit from the 
instruction. Similarly, the finding of no significant effects of learners’ statistical 
background on maintenance posttest scores also showed that the content could be taught to 
statistical beginners. For the transfer posttest, only the courses taken in statistics and the 
self-rating of mathematical skills showed significant effects. This result suggests that 
students having more statistics background and feeling more confident about their 
mathematical skills, transferred the learned skill to the new learning situation better. 

The Dynamic Measure of the Learner's Ability 

According to the results of the covariance analysis, the effect of the dynamic 
measure was consistent across posttests, indicating that the learner's ability to profit from 
the instruction in the learning process could also be a good predictor of hi^er later posttest 
performance. Since the indicator used (the percentage of successful trials) was affected by 
the treatment, the dynamic measure was not directly comparable across the three treatment 
groups. 

Static/Dynamic Measure and Treatments 

The results from the partial correlation analyses indicated the predictive utility of 
static and dynamic measure did vary by treatment, which confirmed other studies' 
conclusions (Bums, 1985; Day & Cordon, 1993). The result from the correlation analysis 
on static and dynamic measures was also consistent with Day and Cordon (1993). The 
results suggest that students' prior knowledge (static measure) might influence the 
students’ ability to profit from the instruction (dynamic measure), especially in the least 
support condition. Because this relationship was not significant in the scaffolded 
condition, and the dynamic measure was the best predictor in that group, the study results 
also suggest that scaffolded instruction could be the better diagnostic method when the goal 
is to obtain a measure of learning capability relatively uncontaminated by existing individual 
differences. 
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Conclusions 

While the term “scaffolding” has been used widely in recent years in the field of 
media assisted instruction to emphasize its support-building component, this study showed 
the importance of its support-fading component. It used a 3-D model to demonstrate how 
the two components interact and compensate each other through the control of the amount 
of support as well as the task. In this study, the 3-D contingent scaffolding model was 
discussed, implemented, and evaluated in a computer-based instruction application, 
“Hypothesis Testing— the Z-Test,” in order to establish baseUne data for integrated-media- 
based instruction and hypermedia-based learning environments. The conclusions are: 

1 . This model can be successful in promoting learning as well as enhancing learners’ 
independent knowledge application. The results of this study not only confirmed the 
problem of learners over-relying on technology support but also showed that the 3-D 
scaffolding model can be the solution to this problem. However, further evidence is 
needed to confiim its superiority on knowledge transfer. 

2 . This model provides a systematic way to link the concept of scaffolding to the 
integrated media design features using both support building and fading techniques. 

The results of this study showed that the computers and hence the integrated media 
could be a good vehicles to employ the 3-D model of scaffolding. 

3 . This model promotes learning consistently across individuals. The dynamic measure of 
the learner’s ability was a better predictor of the learning outcome for this scaffolded 
instruction than were the static measures. If the goal is to obtain dynamic measures that 
predict the learning outcome and that are relatively uninfluenced by pre-existing 
individual differences, the dynamic measure collected during scaffolded instruction may 
be the superior diagnostic method. 

4. This model was implemented well on the topic of hypothesis testing. The results of 
this study also showed that the topic of hypothesis testing could be taught to the 
students with low mathematical background and statistics beginners. 

This study serves as an example of an integrative effort which combined 
constructivist learning theory, statistical problem solving, and CAI design principles. The 
results of the study showed the value of the 3-D scaffolding model on CAI design in 
helping college students construct, test, and refine their understanding of complex statistical 
reasoning. A similar model could be adapted to other integrated media learning 
environments, to various disciplines, and to different audiences. 
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